Method of automatically replicating data objects between a mobile device and a server

ABSTRACT

Network operators can control how data replication services use available bandwidth, in order to make the most efficient usage of that bandwidth, using parameters applied to a data object to be replicated. The parameters may be both time dependent and also relate to how urgently that object needs to be replicated. A change log lists all objects at the device and/or server to be replicated and the parameters then comprise a weight associated with each object that defines how urgently that object needs to be replicated; the weight of each object is then locally compared to a threshold at a given time and the outcome of the comparison determines whether the object is sent for replication or not at that time. This combination of weight and threshold gives a flexible way to control the timing of data replication and hence make the best use of bandwidth.

FIELD OF THE INVENTION

This invention relates to a method of automatically replicating dataobjects between a mobile device and a server; data replication isneeded, for example, to back up data from the mobile device to theserver and to ensure that the mobile device has the most up to date dataheld on the server. The mobile device and the server are connected overa wireless network, which may comprise a wide area network such as acellular telephony network. The wireless network may also comprise ashort range network, such as an 802.11 network, or a combination ofshort range, wide area and wire based connections.

DESCRIPTION OF THE PRIOR ART

Data replication between mobile devices (such as mobile telephones,smart phones, communicators and other kinds of wireless informationdevice) and servers has attracted considerable attention. Reference maybe made to PCT/GB2002/005308 and PCT/GB2002/005311 (to the presentapplicant), the contents of which are incorporated by reference.

One characteristic feature of wireless networks is the need to usebandwidth efficiently; this applies especially to wide area networkssuch as cellular networks, but also applies to other kinds of wirelessnetworks, such as those based on 802.11 systems. The present inventionis directed to increasing the efficiency of use of all such networks.

Cellular telephony network operators currently push data to mobiledevices to update those devices with data, such as menus of the names ofgames that can be download to the mobile telephone. A simple example ofefficient bandwidth useage is the practice of downloading these menus ofgames over night to make use of the available bandwidth.

To date, designers of data replication systems (as opposed to simpleone-way push updating systems) have not been preoccupied with makingefficient use of network bandwidth. This bias arises because theassumption behind most data replication systems is the need forimmediate replication, irrespective of the impact on bandwidth.

SUMMARY OF THE PRESENT INVENTION

In a first aspect, there is a method of automatically replicating dataobjects between a mobile device and a server, connected together via awireless network, in which the timing of data replication across thenetwork is determined by a network operator applying parameters thatmake efficient usage of network bandwidth.

Hence, the present invention is based on the insight that the assumptionthat data replication must be immediate to be of value is flawed.Instead, it is useful to be able to provide network operators (such asthose providing cellular services, or 802.11 services) with the abilityto control how data replication services use available bandwidth inorder to make the most efficient usage of that bandwidth.

The parameters applied to a given object may be both time dependent andalso relate to how urgently that object needs to be replicated. Forexample, a change log may list all objects at the device and/or serverto be replicated and the parameters then comprise a weight associatedwith each object that defines how urgently that object needs to bereplicated. The parameters may further comprise a threshold that is afunction of time, with the weight of each object being locally comparedto the threshold at a given time and the outcome of the comparisondetermining whether the object is sent for replication or not at thattime. This combination of weight and threshold gives a flexible way tocontrol the timing of data replication and hence make the best use ofbandwidth.

Further aspects and details are defined in the appended Claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be described with reference to theaccompanying drawings, which show graphs of how the parameters used tocontrol data replication can vary over time.

DETAILED DESCRIPTION

The present invention is implemented by Cognima Ltd (London, UnitedKingdom) to allow mobile network operators to control the timing of datareplication in the Cognima Replicate™ system. The present documentassumes a working familiarity of the Cognima Replicate™ system, which isdescribed in more detail in Appendix 1

It should be noted that the term Quality of Service, or QoS, is usedthroughout the present document, but unless stated in context this isnot related to the technical meaning of QoS in the context of RFC2475.This IETF document defines QoS very precisely in terms of a number ofmetrics in the IP layer, whereas the first implementation of Cognima QoSwill be applied at the application layer and will not rely uponservice-specific configuration of network server parameters.

1. Scheduled Replication of Data

This invention defines a way in which data transmission across apacket-switched wireless network can be intelligently scheduled, toimprove efficiency of network bandwidth usage without seriouslyimpairing the user experience.

1.1 Cognima Replicate

The Cognima Replicate system is designed to replicate user data betweena mobile client and a network-based server, with zero user intervention.Both the client and the server recognise when there have been changes tothe data which require replication to take place, ensuring the integrityof the distributed object database that is shared by the client and theserver. This approach creates an experience for end users of the system,of always having instant access to all their data, without the any needfor manual synchronisation. An implication of making replicationinvisible to the user is that there need be no indication, and no usercontrol, of when the replication is taking place. The client and serverdevices are in control of when replication should take place, withtiming decisions based on dynamic parameters that can be set by themobile network operator.

1.2 Scheduled Replication and Network Operators

Network operators wish to smooth the peaks and troughs of the dailynetwork usage cycle in order to make most efficient use of thebandwidth. This means moving data traffic away from peak times, andwhere possible moving it into troughs in the cycle. Operators will valuethe ability to tweak settings that affect when replication occurs, andthereby refine network efficiency.

Operators will also wish to offer services of different QoS and costlevels (usually expressed as a choice of service level bundles withtheir existing data tariff) to address the varying demands of customers.This is enabled by providing the network operators with the opportunityto set dynamically, the parameters which define replication QoS from theusers' perspective.

Most importantly the services that Operators offer should provide acompelling user experience to their customers.

1.3 Scheduled Replication and User Expectations

The Cognima technology presents new mental models for users. Replicationscheduling models, and their corresponding Service plans, must be simpleand consistent to aid user acceptance. Users should be shielded from thedetails of replication as much as possible. Data should replicateaccording to users' expectation.

Users want to be able to choose one QoS level from a range of options;otherwise they feel that they are either paying too much or not gettinggood service. The QoS options to the user should be simple. Users willfind difficulty in weighing up the relative benefits of a plan thatoffers Contacts in 2 minutes, Photos in 3 hours, and Banking overnightagainst one offering Contacts immediately, Photos overnight, and Bankingin 30 minutes etc, however much these options might fit the NetworkOperators' demographic research.

Users will feel more comfortable choosing a general standard ofservice—e.g. Basic—and upgrading perhaps one particular service. Userswill appreciate the opportunity to temporarily upgrade the replicationlag, by service or individual object for a cost. For example, users maywant a premium “Send this photo now” option, which would override thedefault priority with which all other photographs are replicated to theCognima server.

1.4 Scheduled Replication and the Cognima Architecture

From an engineering point of view, solutions should be based on simplemodels rather than complex rule-sets. Rules that depend on time-outs(e.g. each object should have its own time limit that starts countingdown when it enters the Change-Log) will severely affect performance.Likewise a solution requiring the CRE to poll the Change-Log every xseconds will also reduce performance.

2. QoS Implementation

2.1 The QoS Profile

We introduce the concept of a QoS Profile which determines when objectsof a given type will be replicated for a given user.

The Network Operator can influence the timing at which all objects arereplicated according to the peak and off-peak tariff or periods of highnetwork traffic demand. It is possible for a Network Operator to definea timing profile for each application, against which each devicereconciles the replication priorities and time limits of objects in thechange-log to determine replication behaviour. The shape of this graphwill be determined by a number of factors including the experience ofthe network operator in monitoring data traffic volumes.

The Network Operator can also influence whether other (lower-priority)objects in a change-log at the time of an open data connection should bereplicated, once the connection-initiating object has been sent. It ispossible to define an opportunism threshold to control this.

E.g. several non-urgent items are in the device's change log. The userchanges a device setting that immediately initiates a data connection.The NetOp has specified that any other object in the change-log under 20kB should be replicated using the open connection.

A different opportunism threshold should be used if the device isoperating in a roamed network, as the cost to the user of initiatingconnections may outweigh the impact to the NetOp of sending more data atpeak periods. The opportunism threshold, along with the other QoScontrol parameters on the client device, are communicated to the clientusing the replication framework and are therefore kept automatically instep between the client and the server.

In a more advanced implementation of QoS control, the network-basedserver will be able to determine the cell loading for a given mobiledevice. When the cell loading drops below a defined cell loadingthreshold, the server should be able to signal the client that it maystart opportunistic replication. This delivers most benefit in all-IPnetworks where the client device has a permanently assigned IP addressand can therefore be contacted instantly by the server. Otherwise theprocess of sending a Communications Initiation Request to the clientdevice influences the cell loading, and adds a latency to the systemthat creates an opportunity for the cell loading to change beforereplication can start.

The Network operator can update the data traffic graph, opportunismthreshold and cell loading threshold after deployment. This allowsoptimisation of the QoS control in the light of experience.

All objects joining the device change log are currently subject to thePause Before Connect delay. However as a Network operator definedmechanism for defining to what extent changes are batched together, thePause Before Connect delay will be largely superseded by QoS control.

2.2 The Service Provider Control

The Service Provider has the opportunity to determine a QoS Profile foreach Cognima service. This profile contains sets of replication timelimits within which Cognima objects created by that service are intendedto be replicated. The actual time at which replication is attemptedwithin the time limits is determined by a number of elements in thesystem including the state of the Cognima client software and the clientdevice, and network factors such as data tariff peak/off peak times,cell loading and so on. The Service Provider can set a zero timelimit—i.e. request immediate replication for objects of a given type. Ifreplication is attempted and fails, the object remains scheduled forreplication but is subject to the existing backing-off behaviour.

E.g. Provider specfies that replication of new contacts to the Servershould be attempted within a 2 hour time limit. If a contact is createdduring a network trough or while the cell loading is very low, thedevice may send the contact immediately. If the timing coincides with anetwork peak or the cell loading is high, the device can wait for up to2 hours for conditions to change, but after this time it must attemptreplication regardless.

The QoS Profile defines the following factors as parameters determiningreplication timing:

Object type (eg. Contact, Photo, DCC director, DCC usage log etc)

How object was created (eg. new contact on handset, edit to contactcreated on portal etc.)

Direction of travel (eg. edits to contact on portal go immediately,edits to contacts on phone go within 2 hours)

Handset is on home/roamed network (e.g. photo replicates immediately athome, but within 12 hours if roaming)

And optionally . . .

Size of Object (e.g. contact is over 5 kB (i.e. contains an image)—sendwithin 12 hours, contact is under 5 kB—send immediately)

Different behaviour when service is first activated (E.g. initial uploadof photos is immediate, thereafter within 12 hours)

The time Emit for replication is assigned to an object by a QoS Profilebased on its time of creation. However it is also possible to change alimit already assigned according to subsequent events such as the memoryavailable on device changing, or if the handset roams to anothernetwork. This creates a need for occasional recalculation of the weightsof the items in the changelog.

The Service Provider is able to define a shelf life for objects in theQoS Profile. If an object reaches its shelf life while it is still inthe change log, the object should be deleted.

E.g. The Service Provider specifies that a weather update in the DCCservice has a shelf-life of 24 hours. If the item hasn't replicated to ahandset within this time, it is deleted from the change-log

The Service Provider can define an object as over-writeable. If a newobject enters the change log that replaces an earlier version still inthe change-log, the earlier version is deleted. In this situation, theService Provider can determine whether the new object should adopt thetiming of the object that has been overwritten, or should enter thesystem with a new replication time limit. The default setting is thatnew objects adopt the timing characteristics of the objects theyreplace.

E.g. The Service Provider specifies that a objects of the weather updateclass are over-writeable, and that new entries should adopt the timingof the ones that get overwritten. This will mean that an old forecastmessage gets overwritten by a newer one, but the newer one is not heldback from replication.

Service Providers can provide different classes of service within theProfile set. One way of dong this is by defining individual QoS Profilesfor each class of service.

Eg. Contacts in a Gold service are replicated immediately aftercreation, but under the Silver service on the same network, replicationmay take up to 2 hours.

A QoS tariff applies to an individual Cognima service, although it maybe presented to the subscriber as a bundle of services at a particularprice point.

A QoS Profile can be changed by the Service Provider once it isdeployed; changes to the Profile are replicated to the relevant clientdevices so that the understanding of each QoS profile is common betweenall entities in the system.

2.3 Notes on Deriving a QoS Profile.

The parameters in a network operator (or ‘netop’) defined profile shouldaim to balance:

What is acceptable user experience for given tariff?

What scope should be given for batching changes (typically more batchingwhile roaming)

What scope should be given for dodging network peaks

It should be expected that users will generally need changes made on aportal to be swifter than those made on the phone—we should expect usersat a PC to have their phone with them, but not vice versa. Initialreplication on activation should be immediate for the best initial userexperience.

The cost to the user of opening several data connections when roaming islikely to outweigh the impact to the NetOp of sending more data at peakperiods.

3. Algorithmic Implementation of QoS

3.1 Introduction

In the Cognima system, QoS is implemented as an increment to thefunctionality of both the client and the server. In particular itrequires a modification to the behaviour of the change log, andintroduces a requirement to recalculate certain attributes of queuedchange log entries. Replication can then occur as a result of theoutcome of this recalculation.

3.2 Algorithm Description

The algorithm is made up of several components. We introduce a changelogitem weight. This weight indicates how urgently a changelog item needsto be sent; the heavier the weight, the more urgent the item. Weintroduce a changelog threshold. Any items that have a weight equal toor greater than the threshold need to be sent immediately. When aconnection is made all items with weight greater than the thresholdminus a delta are sent. The delta represents opportunism.

For the sake of concreteness we say that weight and threshold can varybetween 0-100. The weight of an item that absolutely has to be sentright now is 100; a changelog threshold of zero indicates that anyentries in the changelog can be replicated right away.

Both weight and threshold can vary over the course of a day. There willbe predictable variation and also dynamic variation. Some examples willclarify this.

In FIG. 1, the straight line at weight value 40 shows that the weight isconstant over time. The weight of an item that has to go within acertain time is shown at FIG. 2. The weight of the item starts at arelatively low value, indicating low priority, and then it jumps as wereach the time limit—the new value of 100 will force the client toattempt replication of the object at time T1.

An item that should only go at a certain time will look like the FIG. 2diagram but the jump comes at a particular time, not after a particularduration. The weight of an item might change dynamically. Say forexample that the device starts to run out of space: the weight of itemscould be increased to get them out of the changelog and allow ghosting(see Appendix one to the current description for an explanation of thisterm). The threshold will also have a graph over time, the followingFIG. 3 graph shows an example of how a daily threshold cycle mightappear, with a high threshold to guard against low-value traffic duringpeak hours (e.g. after 9.00 hours for several hours), and a lowerthreshold when the data networks are traditionally quieter (between24.00 hours and 6.00 hours, the threshold is zero).

The example effectively shows a daily cycle split into three tariffbands, perhaps named off-peak, standard and peak, with the replicationthreshold set appropriately for each band. Note that there is a fourthband just after midnight, where the threshold drops to zero—this isintroduced to ensure that all changelogs are emptied once per day duringthe quietest period. This zero threshold period could be defined onceper week or at some other interval defined by the network operator, butis recommended to ensure that the defined QoS profile does not preventsome objects from being replicated at all. As for the other bands in theprofile, the off-peak period extends through the late evening and night,and represents the times at which the data network can expected toexperience low traffic; replicating large objects during this time willallow the mobile network operator to make best use of the limitedbandwidth available during peak times.

The zero-threshold period can be adjusted by the network operator in thelight of experience; finer adjustments can be made for different usergroups, per-application or even per-user thereby staggering the off-peaknetwork usage and ensuring that the full user base for a given serverdoes not attempt to connect to the Cognima Server at the same time.

There will be dynamic changes to the threshold, for example from cellloading or to support a marketing drive promoting a new service (duringwhich time it may be preferable to allow basic tariff data onto thenetwork during peak hours to encourage uptake). If it is possible forthe device to detect that its cell is not busy it could drop itsthreshold a bit which might trigger some replication.

The core of the algorithm is to calculate the threshold graph and theweight graph of every item in the change log. If the current weight ofany item is greater than the current threshold then a connection isestablished. Otherwise the next time that any item will exceed thethreshold is deduced (ie when the graphs intersect) and a timer is setfor this interval. Because both the weights and the threshold can bedynamic there are several events that can trigger a recalculation:

A new change log item is added

The server may push a new threshold value to the client. This isactually a special case of the previous event since the QoS object onthe client is controlled through replication in the normal way, meaningthat a new threshold value will be delivered to the client by placingthe change in the device's changelog queue on the server—this changelogentry must have a weight of 100 to force immediate replication, and theresulting change of threshold may trigger or delay replication of otherentries already in the queue.

A timer expires—this will usually be the timer indicating the point atwhich the weight of an existing entry in the changelog is due to exceedthe current changelog threshold.

The cell (or network) loading changes

The memory available on the client device falls below a certain level

The device detects that its roaming state changes

A new application is deployed and activated on the device

Connection terminated—this also results in the creation/update of the‘next connection’ timer.

There are two separate calculations: the weight graph of an item and thethreshold graph.

Parameters that can affect the weight of an item at any given point intime are:

Direction (client−>server or server−>client)

Shelf life (this is usually encoded in the class)

Overwritable (this is usually encoded in the class)

Size in bytes

Time entered into change log

Priority

Time out interval

Assigned time for replication

User assignment of a non-default priority to a given object (such as the‘send now’ option on an image for uploading to the user's media galleryaccount)

Memory available

The extent to which these parameters influence the weight is controlledby the service provider (i.e. network operator) in the control asdescribed in section “Service Provider Control” above.

Parameters that can affect the current threshold value of a change logare:

Time of day

Roaming status

Cell/network loading

Time since last replication

User tariff

After each refresh of the changelog, the client software also calculatesthe time interval to the next intersection between a weight graph andthe threshold graph; this is so that scheduled replication can takeplace as required even if there are no further changes before thatscheduled time. Generally, the ‘next event’ for the change log willeither be as predicted by this intersection calculation, or will becreated by some external event that places a new entry in the change log(which may of course force a refresh of the values of all weights andthe threshold). Note that the ‘next connection’ event may have a valueof ‘never’ if the changelog is empty, unless the active QoS Profile hasa zero-threshold period as in the example above.

3.3 Banding

We control both the client weight graph and the threshold graph by astructure we call banding. We assume that all graphs can be described assets of horizontal lines which jump (concatenations of step functions).Therefore the QoS Profile for a given class can be described by an arrayof bands. A band is parameterised as follows: Parameter ValuesDescription Band type Delta; absolute Whether the band is measured fromcreation time or against system time. Direction Server->client; client->Direction of travel of item. server Start time Time If delta type aduration, if absolute then a clock time. End time Time If delta type aduration, if absolute then a clock time. Weight 0-100 Item urgency.

Bands can be defined as deltas from a start time or against the systemclock (absolute). A weight graph for a class which should be scheduledto go within 2 hours of object creation could be described by a pair ofbands: The following table could describe the QoS weight profile for theContacts class. Band Band Name Type Direction Start Time End Time WeightBand1 Delta client->server 00:00 02:00 25 Band2 Delta client->server02:00 24:00 90

A threshold graph which describes the graph in the example above wouldbe represented as follows: Band Name Band Type Direction Start Time EndTime Weight Band1 Absolute client->server 00:00 02:00 0 Band2 Absoluteclient->server 02:00 07:00 20 Band3 Absolute client->server 07:00 09:0050 Band4 Absolute client->server 09:00 11:00 80 Band5 Absoluteclient->server 11:00 18:00 50 Band6 Absolute client->server 18:00 24:0020 Band7 Absolute server->client 00:00 24:00 50

Note that there is a different weight for objects created or modified onthe server and that this weight is constant throughout the daily cycle.This approach can be used to mirror the fact that server-side changesmay have a different priority to those originating on the client.

The client and server carry QoS objects which encapsulate the abovetables and influence the scheduling of replication. A user's device willhold a QoS object for each data class installed on the device, plus asingle threshold object representing the whole system. Each QoS objectholds an array of bands to describe a single weight graph. Theintersection points of these graphs determine when objects of a giventype will be replicated—these intersection points must be calculated bydetermining the weight of each object and the threshold value for thesystem at a given time.

3.4 Changelog Item Weight Calculation

The weight of an item in the change log has to be calculated. The QoSobject for the item's class is retrieved from the data store. Thebanding structure is examined and a weight is looked up from the weightgraph (either by calculating current time—created time in the case ofdelta band type, or by comparing system time in the case of absoluteband type). To include object size as a parameter in weight calculation,we have added two more fields to the banding structure. Parameter ValuesDescription Size limit Size in bytes Object size limit over which theoversize weight applies. Oversize weight 0-100 Weight of oversizeobjects.

If the object size is greater than the size limit then the oversizeweight is used—note that this could be higher or lower than the defaultweight for the class. In some applications, the object weight can beoverridden by a user request (effectively a ‘send now’ button) whichsets the weight to 100.

3.5 Changelog Threshold Calculation

The current changelog threshold weight can be extracted from thethreshold QoS object using the current system time. This value can thenbe modified by dynamic variables—for example if the device can detectroaming status then this can influence the threshold. Generally thethreshold will be higher when roaming, to reflect the fact thatreplication will be more expensive; it is also possible to specify alower limit for the threshold in a given band, effectively preventingvery low priority objects from replicating at all before the devicereturns to its home network. In deployments where the mobile terminalcan be aware of local cell loading conditions, then the cell loading canbe used as a factor in adjusting the current threshold value: if thetraffic loading of the local cell is below some value (e.g. 70%) thenthe threshold can be reduced. If the cell loading is above some value(e.g. 95%) then the threshold can be increased.

The QoS object creates timer events which represent the times of day atwhich the threshold is known to change, reflecting the shape of the QoSProfile as defined by the network operator. As each of these timerevents fires off, the QoS threshold will be assigned a new value and theensuing weighting recalculation will allow objects with the correctweight to be replicated.

3.6 Next Connection Time Calculation

The weight graph of an item needs to be compared to the change logthreshold graph to find the next time that the item weight>=thresholdweight. This calculation will ignore dynamic change to the thresholdweight which is by definition unpredictable. It is a fairly simplecalculation to match up the bands and compare weights at consistenttimes.

There are three types of event that might result in a replicationsession:

a change object already in the changelog may increase in-weight due tomoving into a new band

the QoS threshold may drop due to a move from one time-based tariff toanother

a new entry may appear in the changelog with a weight higher than thecurrent threshold.

For the first two events, the time at which the session starts ispredictable and must be calculated at the end of each replicationsession. This represents the next scheduled connection time.

Appendix 1 Data Replication System Description

The present invention will be described with reference to animplementation from Cognima Limited of London, United Kingdom. Cognimahas developed a data replication technology that directly addresses theneed for Mobile Service Providers (MSPs) and Network Operators toincrease consumer adoption of data services, encourage greater loyaltyfrom their valuable customers, and differentiate their services from thecompetition.

Cognima's data replication solution addresses these issues by:

Increasing adoption by making data services compelling and effortless touse.

Establishing a high barrier to churn by securely backing up subscribers'personal data on servers controlled by those subscribers' MSP.

Enabling the MSP to create differentiated services by controlling thecustomer experience.

1. Overview of Uses for the Cognima Data Replication Framework

Cognima's data replication framework enables a Mobile Service Providerto build compelling services for consumer markets. The MSP hosts aCognima Server at its data centre. The server comprises an Oracledatabase plus Cognima's multi-threaded Java communications server,hosted on a standards-based J2EE application server and carrier-gradeUnix hardware. Section 4 and later sections describe the technicalimplementation in detail.

The Cognima framework replicates data entered in a mobile phoneautomatically (without any user intervention) to other phones via theCognima Server. Similarly, data from external systems connected to theCognima Server is automatically kept up-to-date on mobile phones.

Mobile subscribers using Cognima-enabled applications experience analways-available, instant connection to their personal information andfiends.

Personal information can include the subscriber's address book,messages, bank account details, stock prices, pizza orders, calendar,current traffic on a route to work, or any other personalised content.The data is always kept securely backed-up on the Cognima Server andautomatically replicated on all relevant client devices.

Always-available means that the personal information is accessible onwhichever device or handset the subscriber is carrying, whethercurrently connected to the network or not since the user can alwaysaccess personal information stored locally on the device). Users canalso edit and manage their personal data-directly on the server via aweb interface—the Virtual Phone.

Instant means that subscribers do not have to wait for data to downloadfrom a server; the latest information is on their handsets even beforethey know they need it since that data is automatically sent to thehandset (e.g. polling by the handset may occur; this can be regularperiodic—such as every 30 minutes or at pre-defined times (4 pm, 5 pmetc). Pushing to the handset may also occur).

Subscribers can share their data across multiple devices and with theirfriends since the Cognima Server can replicate this data to any defineddevice or defined individual.

1.1 Example Cognima Applications Customer Need Cognima Application SarahSarah's phone has been Whenever Sarah enters data in stolen, includingsome her phone, Cognima important contact automatically backs it up on anumbers and messages central server at the MSP's for which she has madedata centre. Sarah can buy a no manual back-up new mobile phone, andcopy. retrieve all her contacts and messages instantly from the centralserver, as long as she remains with the same MSP. She can also deleteher data from the stolen phone via the MSP's portal. Jill Jill is outshopping. Cognima keeps Jill's Before making an personalised contentexpensive purchase, she (including her bank account needs to know if herdetails) up-to-date salary has been paid into automatically on hermobile her bank account. phone by periodically (or at a However, she isin the predefined time or even basement of a immediately a changeoccurs) department store, and sending any changed data to has no networkJill's mobile. The latest data is coverage. there on Jill's phone evenbefore she knows she needs it. She can access it instantly, even ifthere is no network coverage. Matthew Matthew likes to keep Cognimashares Matthew's his friends informed presence profile with his abouthis current friends. When he changes his availability and ‘mood’.profile (e.g. selects an icon to He also likes to see what indicate he'sfeeling sociable) his friends are up to. the icon updates automaticallyHe's mainly interested in in Matthew's address book keeping track ofwhat's entry on his friends' phones. happening in his social Matthew cansee presence group, and he wants to information for all his friends dothis at a glance, at a glance on his own phone. without having to go Hecan even ask his phone to ‘on-line’ or send lots of alert him when afriend is expensive messages. feeling sociable or bored, so that he canimmediately call. Laura Laura has two mobile Cognima automatically keepsphones - one she uses at all the data in Laura's phones work, and afashion- in step. Whenever she edits phone she takes out in data on onehandset, it is the evenings. She wants immediately (or periodically orto keep the same at a predefined time) replicated address book on bothonto the Cognima server devices, but she hates which then updates herother entering data twice, and phone as well. She never has she's neverfigured out to remember to press a ‘sync how to use the sync button’ -it just happens. Jill software that came with even shares some of theher phone. Swapping contacts in her phone with her the SIM card over ishusband, Geoff. When Geoff cumbersome, and leaves enters his mother'snew behind data in the mobile number, it is phone memory. automaticallyupdated in Jill's phones as well. Juha Juha also has two mobile WithCognima, SMS, e-mail devices - a phone and a and other types of messageswireless-enabled PDA. can be read and sent from any He needs to read anddevice, and also using a reply to e-mail and SMS ‘Virtual Phone’ webinterface. messages on both Messages are received on all devices, but hegets devices used by the subscriber, confused and frustrated, and sentmessages appear in and loses productivity, the Outbox on all devices.when his Inbox gets out Any message read on one of sync. device isinstantly marked as read on all other devices. Messages deleted from amobile phone can be stored and retrieved via the Cognima Server.2. Benefits to the Mobile Subscriber

Cognima provides an ideal framework for implementing mass-marketconsumer data services based on the following key benefits:

Friendliness: no user intervention is required. Subscribers never needto press a ‘sync’ or ‘download’ button to access their data. Systemconfiguration and secure data transfer are completely transparent to theend user.

Instant availability: the user is always able to interact instantly withlocal data (even when off-line), whilst any updates take place silentlyin the background. For example, users can read their personalisedcontent whilst on an underground train. The user experience is separatedfrom the data transport.

Affordability: The MSP can control when replication takes place, and theQuality of Service (QoS) delivered. However, because the user experienceis separated from the data transport, lower QoS does not affect theuser's perception of the service. Crucially, this allows the MSP tooffer low-cost, subscription-based services with relatively poor QoSwithout sacrificing user experience—e.g. data replication can happenovernight for non-urgent data services such as bank statements, yetstill be satisfactory to users. Overnight data replication usesotherwise underused bandwidth and is hence far cheaper than peak timedata replication. Urgent data replication (e.g. presence information)can happen at any time on a periodic or (optionally) continuous (push)basis and attract a higher charging rate. Furthermore, efficient use ofphone memory & processor power allows Cognima client software to becost-effectively installed in even the cheapest mass-market phones.

3. Benefits to the Mobile Service Provider

Cognima presents a MSP with a means to generate new data revenues,reduce churn, and to differentiate its services from those of itscompetitors.

3.1 Increased Usage of Existing Mobile Services

Cognima increases usage of existing mobile services:

Messaging and content-based services become much more convenient andimmediate, and will therefore be used more.

The enhanced immediacy of presence information increases the use of chatand Instant Messaging, and an alert when free capability will boostvoice calls.

Effortless management of multiple devices allows users to carry anappropriate phone on any occasion, and therefore make more calls andsend more messages.

3.2 Compelling New Services

Cognima enables rapid introduction of compelling and affordable newmobile data services.

Cognima delivers a compelling user experience for new services inlow-end phones using only spare network capacity. This is affordable andscalable for the network operator, allowing the MSP to offerunderstandable and predictable pricing for mass-market subscribers.

Most of the application development for new Cognima services takes placeon the server side, allowing the MSP to bring new services to marketquickly.

Cognima's client software can be installed as a flash memory upgrade,endowing today's mass-market handsets with smart-phone-likecapabilities. New software applications can be downloaded over the airto existing Cognima-enabled handsets, allowing MSPs to roll out new dataservices without waiting for new devices to support them.

Third party application developers can leverage the MSP's Cognimainfrastructure to develop new applications for the MSP's network.

3.3 Churn Reduction

Cognima services act as a significant barrier to churn. For example, asubscriber who stores their personal information securely at their MSP'sCognima Server can buy a new phone and immediately retrieve all personalinformation to their new device. All this personal information may belost if they decide to take out a subscription with a different serviceprovider.

3.4 Differentiation

Today, subscribers have the same basic experience of using mobile dataservices on all networks. For example, the experience of using WAPservices is defined by the WAP protocols, the browser in the phone, andthe content accessed. Many MSPs have realised that they mustdifferentiate themselves by giving their subscribers a unique userexperience, but are hindered from doing so by severe constraints tocustomising the services in mobile handsets.

Cognima gives MSPs the ability to implement services on the handset, andthereby to regain control of their subscribers' user experience. Mostimportantly, Cognima allows this without sacrificing interoperability;support for industry standards is achieved through straightforwardintegration with the Cognima Server. The net result is that the MSP'sposition in the value chain is strengthened versus the powerful brandsof handset manufacturers and content providers.

4. Cognima Data Replication Framework Functional Design

4.1 Introduction

This and subsequent sections of the Detailed Description are intended todescribe how the Cognima data replication system actually works. Itcovers the behaviour of client devices, the Cognima Server and the webclient, without going into details of specific hardware, programminglanguage, software class design or environment It does describe thebasic data structures and algorithms used. Terms Client device A phone,PDA or other machine running the Cognima client software. Cognima Aserver accessible by client devices which runs the server Cognima serversoftware to replicate data. Replication The process of copying data froma client device up to the Cognima Server and then down to other clientdevices belonging to the same user. User A human being who owns and usesat least one Cognima client device User data The set of information(contacts, messages, ringtones, pictures etc) that a user might want tostore and manipulate on a client device.4.2 Purpose

The objectives of the Cognima software are:

To allow a user instant access to view and modify an ‘up to date’ copyof their data on multiple handheld devices capable of wireless dataconnectivity.

To allow a user to view and modify the same data using a conventionalweb browser.

To effortlessly provide secure backup of a user's data.

To give a user powerful data functionality on a cheap handset bydisplacing complicated and expensive processing to a server.

4.3 Highest Level Description

Client devices hold a copy of the user's data in a database on theclient device. The user can access this data whether or not he has anetwork connection and therefore always has instant access. When a userchanges the data on his device, the changes are copied to a Change-Log.The client device connects periodically to a Cognima Server on thewireless network, to send up the changes from the Change-Log and receivenew data. This separates the act of changing data from the need toconnect to the network (i.e. push is not continuous in a preferredimplementation). The Cognima Server updates its own database with datachanges received from the client device, and populates Change-Logs forany other devices the user owns. When these devices next connect, theywill receive the changes and thus the devices are kept in sync, eachwith a copy of the same data.

The Cognima Server contains a web server which allows the user toexamine directly using a web browser the copy of the data held in theCognima Server database, and make changes to it as he would on a clientdevice. The Cognima Server also acts as a gateway for the user tocommunicate with other servers on the network/internet. For example, theclient device can effectively ask the Cognima Server to send a messageas an SMS or an email or a fax by setting a few flags in a messageobject and the Cognima Server contains the functionality to communicatewith email servers, SMS servers and fax machines. This can be extendedto servers holding ringtones, banking details, games etc. It is easierand cheaper to build the software on the Cognima Server to talk to theseother servers, than it would be to build the software on the clientdevice.

5. Lower Level Concepts

5.1 Data Structures

5.1.1 Ids

Cognima user data is described using the terminology of objectdatabases: classes and objects. Unfortunately, there is room forconfusion with similarly named OO programming concepts and caretherefore needs to be taken.

All users in a Cognima network are assigned a user id. This id is uniqueto the network—i.e. provided by a given network operator. All users havea Cognima address which is a combination of their user id and CognimaServer URL. This is unique in the world. Each device which belongs to auser is assigned a device id. The device id is unique to the user. Thisis only 8 bits so a user can have a maximum of 253 devices (id 254 isreserved for the web, id 255 is spare, id 0 is invalid). All user datais classified into classes (contacts class, messages class, banktransactions class etc) and the classes are assigned a class id which isunique in the world. Class id ‘12’ refers to a contact, for example.

An instance of a class is an object, which is assigned an object idunique to the user, e.g. a contacts class object might be the contactfor “John Smith”. The object id is generated by concatenating the deviceid of the device which created the object with a monotonic increasingcount which increases over the life of the device. So each device cancreate a maximum of 16777215 objects (if we encountered this limit wecould reset the device id). Classes are defined by the properties whichconstitute them. A class is essentially an array of properties. Eachproperty in the class has a property id which is unique to the class(and is actually just the array position of the property in the propertyarray, starting from zero).

5.1.2 Creating Objects

An object is created on a device. It is assigned an object id and savedto the device database. A copy is also saved into a Change-Log. When thedevice next connects to the Cognima Server the entry in the Change-Logis sent up. The Cognima Server saves the object to its database(recording the system time), does any class specific processing that maybe required (such as generating and sending an email) and adds entriesto Change-Logs for any other devices that the user may own which havedeclared interest in the class. (The entries should be for the correctversion of the class on the device).

An object may also be created on the web portal. The object id isgenerated (using device id of 254 as described above) and processedidentically to the device. There is no Change-Log for the web portal, itgets selections directly from the Cognima Server database.

An object may also be created by a server application (e.g. a messagingmodule might receive an email from which it creates a message object).The object id is generated (using device id of 254 as described above)and processed identically to the device.

5.1.3 Updating Objects

One or more properties of an existing object are modified on a device.The changes are saved to the device database. Each changed property isused to generate an entry in the device Change-Log. These are sent up tothe Cognima Server.

If the time of the update is later than the ‘last changed’ time for theproperty in the Cognima Server database then the Cognima Server savesthe changes to its database (recording the new ‘last changed’ time forthe property), does any required class specific processing and addsentries to Change-Logs for other devices which belong to the user, havedeclared the class and have a version of the class which contains theproperty. The update is also placed on the Change-Log for the devicethat originated the change. This may seem strange but is required tocope with the following scenario:

A user has 2 devices A and B. He updates property 7 on A offline at 5 pmand updates it on B offline at 6 pm. He connects to the network with Afirst. The value of 7 on A gets put in the Change-Log to be sent to B.Later B connects. Its value of 7 is more recent so the value of 7 on Bis sent to A, but B gets A's value. Replicating the value of 7 back to Bfixes this.

If an update is received by the Cognima Server for an object which ismarked as deleted and the update is later than the deletion, then thisis interpreted as an un-deletion. The object is undeleted, updated andthen a refresh of the object in placed on the Change-Logs for allappropriate devices. Updates from the web portal or server applicationswork in the same way.

5.1.4 Deleting Objects

An object is deleted on the device. It is removed from the devicedatabase and an entry is put on the Change-Log listing the class id andobject id. The entry is sent up to the Cognima Server.

If the time of the deletion is later than the last updated time of theobject, then the Cognima Server marks the object as deleted in itsdatabase, does any class specific processing and adds the entry to otherdevices that belong to the user and have declared the class.

If the time of deletion is earlier than the last updated time then thisindicates that the deletion is invalid and a refresh of the object isput on the Change-Log for the device which originated the deletion.

The deleted object is viewable in the web portal a manner that makes itsdeleted status clear. The user can select the object for un-deletion.The deletion mark is removed from the object in the Cognima Serverdatabase and entries to refresh the object are placed on the Change-Logsfor all devices that belong to the user and have declared the class.

5.1.5 Property Types

Each property has a type. There are currently 9 permitted propertytypes: Type Type name value Type description KcogTypeRef 0 4 byte objectid of another object KcogTypeInt 1 signed 4 byte integer valueKcogTypeUInt 2 unsigned 4 byte integer value KcogTypeFloat 3 signed 4byte floating value KcogTypeStr 4 a CogString (a 4 byte unsigned integerholding the number of characters in the string, followed by thecharacter bytes) KcogTypeTime 5 unsigned 4 byte integer value indicatingthe number of seconds since midnight 1st Jan 1990 KcogTypeTypedStr 6unsigned 4 byte integer value followed by a CogString KcogTypeBlob 7 astream of bytes preceded by a 4 byte unsigned integer which holds thenumber of bytes KcogTypeArray 8 a blob structure which can hold an arrayof any kind of data

A CogString is a character count followed by the characters. If thestring is ASCII then the space taken up by the string will be (4+charcount) bytes. If the string is Unicode then the space taken up will be(4+(char count * 2)) bytes.

A CogTypedString is a CogString preceded by a type (4 byte unsignedinteger). The only use of a typed string so far is a Contact Point. Thetype identifies the type of contact point (e.g. email address, homephone) and the string holds the address (e.g. bob@xxx.yyy, 01233556677).

A CogBlob is a length in bytes followed by that number of bytes. It canbe used to store any binary data.

A CogArray is passed around as a 4 byte unsigned integer ‘type’ followedby two blobs. The ‘type’ indicates the type of elements held in thearray. The first blob is an index blob: it holds a sequence of offsets(4 byte unsigned integers) into the second blob. The second blob is thedata blob which holds the elements of the array as a sequence of binarylumps. Elements can be extracted from the data blob by counting alongthe index blob to get the offset of the start of the element in the datablob. This is the stream structure of the CogArray as it is passedaround. Inside a particular system it may appear as a conventionalvector (i.e. already parsed).

The only implemented example of a CogArray is the MessageAddress. Eachelement of the MessageAddress is an AddressPair. An AddressPair is acontact id (object id of a contact object) followed by a Contact Point.

5.1.6 Smart Property Parameters

Some of the properties can be made “smart”. This means they can beparameterised for a specific device to sculpt the data in the propertyfor the characteristics of the device. In practice the parameters aretwo 4 byte unsigned integers, one is a smart type and the other is a maxsize. For example, the property which holds the body text of a messagemight be parameterised to smart type kCogPlainText and max size 100 on acheap phone with limited memory, but parameterised to be smart typekCogRichText and max size 1000 on a PDA with more memory.

The parameters are stored by the Cognima Server when the application isadded to the device. When new objects or updates for that class areplaced in the Cognima Server Change-Log for that device they areprocessed according to the. smart parameters. This might involve, forexample, truncating text, converting Unicode text to narrow text orconverting image formats.

It is important for data integrity that the object held in the CognimaServer database be a copy of the object as it was generated. Even if yousee a cut down version on a device you can effectively manipulate thecomplete version on the Cognima Server.

5.1.7 Class Versions

We have the concept of a class version which is defined by a 4 byteunsigned integer. A new class version may add properties to the end ofthe old class, but it may not change or remove existing properties, orinsert new properties between existing properties. This should allowinteroperability between versions. Class definitions with differentsmart property parameters are not different versions.

5.2 Passing User Data Around

Cognima utilises the idea of class metadata to minimise the data thatneeds to be copied around between databases. Class metadata isessentially an array of property metadata. Property metadata is aproperty id, a property type, a smart type and a max size.

User data is transferred as a stream with no formatting informationother than a class id. This stream is parsed by looking up the classmetadata. So if a stream is received for class 6 and the class metadatafor class 6 says that property 0 is a KcogTypeUInt and property 1 is aKcogTypeStr, then you know that the first 4 bytes of the stream shouldbe interpreted as an unsigned integer, the next 4 bytes should beinterpreted as an unsigned integer holding the number of characters n inthe succeeding string, the next n (times 2 if Unicode) bytes hold thecharacters in the string etc.

Client devices declare to the Cognima Server the classes that theysupport. This enables the device to subsequently send up only raw userdata (with a header containing class id, object id and a few otherthings) and hence minimises bandwidth requirements. This can becontrasted with, for example, XML reliant systems that are far morebandwidth hungry.

The client device class declarations also contain the smart propertyparameters so that the Cognima Server can sculpt the data for thedevice. It is worth emphasising that the meaning of a property is hardcoded into an application. The class metadata states that property 2 inclass 7 is a string with max length 30 characters. It is the code in theapplication that interprets property 2 in class 7 as the name of afootball team.

5.2.1 Data Replication Issues in More Depth

Data is held in objects that are created on client devices and theserver these devices connect to (known as the Cognima Server). Theseobjects and any changes made to them are replicated between the clientdevices and the Cognima Server.

The design of the replication process allows:

A set of objects to be defined that will be replicated so that the sameset of objects will be held on a Cognima Server and all the clientdevices that are logged on to that server for a given user. New objectscreated on any device or the server will be replicated to all otherdevices. Changes in any property of an object will be replicated to alldevices.

Only the minimum data to be transmitted across the network for a givenupdate since only changes in data are sent from clients to the CognimaServer or vice versa.

A key part of the design was to not require times of modification to bekept for each property of an object on the client device as updatingthese on constrained client devices is slow and keeping a last modifiedtime for each property in an object would take a lot of space.

On the Cognima Server storing modification times for all properties ofan object is fine as the server has enough storage space and processingpower to deal with this.

5.2.2 Metadata

In order for the system to work it needs a clear idea of what propertiesare defined for a given class of objects. This is done by providing theprogrammer with a few C++ compiler macros that allow definition of theclass metadata.

The definition of the properties to be used in a class result in a ClassMetadata definition. This definition tells the CRE (Cognima recognitionengine) what type a given property is and allows it to pack and unpackan object or a property for transmission over a data link. In order forthe CRE system to work all clients and the server must have the sameclass metadata definition. Thus the following occurs:

When a new Metadata definition is declared on a client device it is sentto the Cognima Server and from there the Cognima Server will send it toall other clients.

When a new Metadata definition is declared on a Cognima Server thedefinition is sent to all client devices.

When a new client device logs on to a Cognima Server for the first timeall of the metadata definitions are sent to that device before anyobjects are sent.

In all of the above cases a future optimisation may be made so that theCognima Server only sends the metadata definition to clients who accessthe class (and the specific properties) the metadata refers to.

5.2.3 ChangeLog

The purpose of the ChangeLog is to record any changes that have occurredsince the client device last connected to the Cognima Server (or theCognima Server to the client device). Using Cognima APIs, applicationsconnect to the CRE and can cause objects to be created or deleted, or aproperty in an object to be changed. These changes are added to aChange-Log on the local device as they are made together with the timethe change was made. Objects are given unique identifiers when they arecreated so that a given object can always be identified.

In the same way, creation and deletion of objects and changes to objectproperties by applications running on the Cognima Server result in thechanges being added to all the Change-Logs of all the client devicesregistered to that user on the Cognima Server. The time of changes arerecorded for each object or property.

ChangeLogs can be built in two ways:

As the new objects are created and properties are changed (this isnormally the case for client devices)

Or they can be built on demand when they are needed by using the lastmodified times of objects and properties if these are stored on thesystem (in some circumstances, this method may be used on the CognimaServer instead of the above method).

5.2.4 Replication

When a client device has items in its ChangeLog to send it will connectto the Cognima Server (and likewise for the Cognima Server connecting tothe client device). By default, the items in the ChangeLog are sent inthe order in which they were added to the ChangeLog, however they may bere-prioritised immediately before sending to provide for premiumservices, urgent data and so on. Items transferred are:

A metadata definition including the type of each property of a givenclass of objects.

A new object that has been created—with the contents of the propertiesof that object.

A property has been changed—with the new value of the property.

An object has been deleted.

In all the above cases the appropriate IDs are sent to identify theobject, class and properties involved. All ChangeLog items are markedwith the time the item was added to the ChangeLog. These times arealways local machine times and are resolved into GMT by the TimeManagement approach described in Section 6.2.

When a client device receives ChangeLog items from a Cognima Server:

When a client device receives a new object message from a Cognima Serverit adds this new object to its local database.

When a client device receives an object deletion message from a CognimaServer it marks the object as deleted in its local database.

When a client device receives a property change it is always assumedthat the Cognima Server is authoritative on the current state of thedatabase and so the change is always made to the value of the propertyheld in the local database.

A Cognima Server receives ChangeLog items from a client device:

When a Cognima Server receives a new object from a client device it isadded to the Cognima Server database and also added to all theChange-Logs of the client devices registered to that user, apart fromthe Change-Log of the machine that sent the new object in the firstplace.

When a Cognima Server receives an object deletion from a client devicethe object is marked for deletion and an object deletion message isadded to all the Change-Logs of the devices registered to that userapart from the Change-Log of the machine that sent the object deletionin the first place.

When a Cognima Server receives a property change it compares the time ofthe change to the current time held for that property on the CognimaServer. If the time of the property change is later than that held onthe Cognima Server the property value is changed in the server databaseand this change is also added all the Change-Logs of the client devicesregistered to that user—including the one of the machine that sent inproperty change (in case another object update has been sent to thatmachine in the meantime). If the property change was not later than theone held on the Cognima Server no change is made as the stored propertyvalue is more recent—but the value is added to the list of old propertyvalues on the Cognima Server so that a user can retrieve it later ifrequired. When times are compared the Time Management approach describedin Section 6.2.below is used.

When a device first connects to a Cognima Server it will be sent allclass metadata definitions and then all the objects in the database forthat user. The Deletion messages generally just mark an object fordeletion. Actual removal of the object from the database may occur lateron once all objects referring to that object have also been deleted.

5.2.5 Optimisations

An optimised version of the above replication protocol allows foraggregation of the entries in the ChangeLog. If a ChangeLog (in theCognima Server or on a client device) has not yet been replicated, and asubsequent entry is added, then existing entries can be scanned topotentially reduce the number of entries that need to be replicatedduring the next connection:

if the new entry is an update to a property that is already scheduledfor update then only the later entry need be retained

if the new entry is an object deletion then all property updates forthat object can be removed from the ChangeLog

if the new entry is an ‘undelete’ command and the original deletion isstill in the ChangeLog then the two entries can both be removed from theChangeLog

6. Core Algorithms

6.1 Handling Endian-ness

Operating systems are fundamentally little endian or big endian which isa choice of the byte order in which numbers and strings are stored. Iftwo computers which have different endian-ness have to communicate thenone of the computers will have to switch the endian-ness of its datapackets. In the Cognima environment the Cognima client software uses thesame endian-ness as the host client device. The Cognima Server has todetermine the endian-ness of the client device (it uses a referencevalue in the first packet of data from the client) and then convert thesubsequent incoming data if necessary to maintain consistent endian-nessin the Cognima Server. The Cognima Server also has to convert anyoutgoing data it sends back to the client device.

6.2 Synchronising System Times

Different devices will inevitably have slightly different system times.Changes that are sent from client devices to the Cognima Server arestamped with the device system time at the time of the change. It is upto the Cognima Server to resolve the times on different devices so thatit can judge the order in which changes took place and record thecorrect update.

The logon of a device contains the current device time. The CognimaServer should be able to compensate for the latency of the network andcompare the login time with its own system time. This will give it adelta between the device time and the Cognima Server time. This deltacan be applied to further times sent up by the device in that session.

The Cognima Server can compare deltas in successive sessions from adevice to determine clock ‘creep’ on the device or changes of time zone:it cannot be assumed that all the client devices in the system haveclocks that are well synchronised to each other:

Clock times drift on devices depending on the device's clock accuracy.

Some users like to set clocks 5 minutes early for example.

Some users will make changes to clocks to account for daylight savingrather than adjusting the locale settings (and some OSes may not providelocale features anyway forcing the user to change the clock directly).

To get round this problem, the server will be responsible for adjustingtimes used by the client device to GMT when comparisons are made on theServer, and from GMT to the equivalent time for the client device whenmessages are sent from the Cognima Server to the client device.

The client device will tag all the items in the ChangeLog with timesobtained from the local clock—as far as the client device is concernedit only ever deals in time based on the client device's own clock.

Each time the client device connects to the Cognima Server it sends itsview of the current time as given by the clock on the client device.From this the Server can work out:

What the delta to GMT is

If there has been any drift in the mobile device clock since the lasttime it logged on since the server keeps a record of the last delta toGMT and when the last connection was made and therefore can comparethese. If there is drift the server can adjust all times sent by themobile device pro-rata.

For example the table below shows a pattern of events with a clientdevice connecting to a Cognima Server. The Client device's time is 5minutes slower that the Cognima Server and is loosing a minute everyhour (an extreme case to show the point). Also to show the point we willassume that from 09:00 to 12:00 the user is on a plane and out ofcontact with the Cognima Server so it does not connect during this time:Client Cognima Server Action Device Time time (GMT) Client deviceconnects to 09:00 09:05 Cognima Server A change is made to property A10:00 X A change is made to property B 11:00 Y Client device connects to12:00 12:08 Cognima Server

In order to work out if the property changes were made before or afterthe time stored on the Cognima Server the times X and Y need to beworked out. From the information above the Cognima Server knows thatwhen the client last connected it was around 3 hours ago and at thatpoint the time difference was 5 minutes whereas now it is 8 minutes.Thus, assuming the clock drift happens linearly, the Cognima Server canwork out that the device is 5 minutes behind GMT and that the clock isdrifting back a minute every hour.

From this is it possible to work out that the time the client deviceknows as 10:00 for the property A change needs to have 5 minutes addedto it for the initial drift, plus one minute for the extra drift thatoccurred in the hour till that property was changed.

Likewise Property B needs to be adjusted to 11:07—the 5 minutes initialdrift plus 2 minutes since two hours elapsed from 09:00 to 11:00 whenthe property was changed.

In practice the delta to the time between the client device time and GMTmay be minutes, but the drift will be in the order of fractions ofseconds per hour.

6.2.1 Time Adjustments

As well as the delta to GMT and any drift in the client device clock,users can also change the time on the client device. They may do this toreset the time to the correct local time (we can give the user theoption to have this happen automatically but some users may want to keeptheir own control of their client device time—e.g. they like to have theclock set 5 minutes fast). They may also make adjustments to reflect achange of local time (i.e. daylight savings or changing timezone). Thegoal is that the user can change the clock on the device to any timethat suits the user and the device simply takes account of this.

When the user makes a change to the client device time most operatingsystems will report this change (for systems that don't do this the timecan be polled say every minute to check for such a change). On detectinga change in time the client device will work out the delta between thenew time and the time as it was before the change. For example this maybe a change of plus one hour as a user moves timezone. The client devicestores this time difference as the Adjust Time which it saves for thenext connection to the Cognima Server. The client device also goesthrough every entry in the ChangeLog and updates all times in the log byAdjust Time. This ensures that the entries in the ChangeLog are alwaysrelative to the local time on the client device.

Several such adjustments could be made between connections to theCognima Server—each time the amount of the time change is summed withthe Adjust Time and the ChangeLog updated so that the times in the logare all relative to the local time on the client device.

When the client device next connects to the Cognima Server the clientdevice sends at logon the stored Adjust Time—i.e. the amount by whichthe client device clock has been adjusted backwards or forwards sincethe last connection. The Cognima Server can then remove this amount fromthe time from the delta to GMT and drift calculation.

6.2.2 GMT to Client Device

The same set of calculations can be made in reverse to convert the GMTtimes of changes made on the Cognima Server to the correct local timefor a given client device.

6.3 Adding an application

An application will use one or more classes to hold user data. Thedefinition of the class is hard coded into the application. The versionof the class is co-ordinated by releases of the application.

Say that a statistics application uses a Footballer class to hold dataabout footballers. When the application starts on a client device forthe first time, it inquires from the device what version of theFootballer class the device already holds. If the version on the deviceis the same as the version that the application has been hard coded touse then nothing more need be done.

If the device holds a newer version of the Footballer class, then theapplication needs to be robust enough to cope with more properties thanit expected. (This situation would arise if you had a class being usedby multiple apps and for some reason you installed an older version ofone of the apps. This should be rare: ideally interdependent apps shouldbe upgraded together.)

If the device holds an older version of the Footballer class (or noversion at all) then the application's version of the Footballer classshould replace it. The new version is sent up to the Cognima Server. TheCognima Server therefore maintains a list of versions of classes used onall devices.

The web portal pages will be the equivalent of the hard-coded deviceapplication. The web can extract objects from the database according tothe latest version of the class, and if there are more properties thanit was hard coded to expect it can ignore them. Therefore the web doesnot need to declare class versions.

6.4 Change-Log Optimisation

The Cognima Server maintains Change-Logs for all devices listing changesthat will be sent to the devices when the devices next connect. Therewill be optimisations that can be made to the Change-Logs, for example:

If>2 updates to the same property are queued in the Change-Log then onlythe last need be kept.

If a deletion is queued for an object then any updates ahead in thequeue may be removed.

If an update is queued for an object then any delete ahead in the queueshould be removed.

If a device registers a new application there could potentially be verymany objects to send down to it (e.g. message history). The Change-Logshould only have a sensible number of objects added to it (e.g. the 20most recent messages).

7. Ghosting, Resurrection, Pinning and Withdrawal

The space available on a client device to hold user data will typicallybe orders of magnitude less than the space available on the server. Thedevice needs to hold a subset of data and the user should have to do aslittle work as possible to maintain this subset. Ghosting and withdrawalare tools to aid this.

A class definition may include flagging certain properties as‘ghostable’. This means that if the object is ghosted those propertieswill be nulled, freeing room on the client device. Ghosting is doneautomatically on the device. The decision about which objects to ghostis made by following a ‘ghosting rule’ and applying the rule whenever anobject is created or updated. The rule defines the maximum number of aselection of objects. When the maximum is exceeded the objects in theselection at the bottom of a sort order are ghosted.

For example, the class might be messages, the selection might bemessages in the inbox, the sort order might be by date/time and themaximum number might be 50. If there are 50 messages in the inbox and anew message arrives, the oldest message in the inbox is ghosted.Ghosting may remove the message body but leave enough-header informationfor the message to be recognised.

Withdrawal (also known in the past as auto-deletion and removal) issimilar to ghosting but works by removing the entire object, not justpart of it.

Neither ghosting nor withdrawal are notified to the Cognima Server. Theyare purely local to the client device. Therefore different devices mayhave different numbers of objects. The data on the devices is stillfundamentally in sync, but the devices hold different data subsets.

If the user wants to resurrect a ghost then a request is passed from theclient to the Cognima Server for the object to be resurrected. A refreshof the object is sent down to the device and the object is put back tonormal.

Individual objects can be pinned. A pinned object is never ghosted orremoved. Pinning can be chosen by the user, or it can happenautomatically. For example, an object that is resurrected isautomatically pinned.

8. User Replication—Sharing Objects

There are many applications for which we envisage it will be useful forusers to be able to share objects. The general way that this will workis: A user needs to know the Cognima address of users that he may wantto share objects with. It is more appropriate to discuss the retrievalof these addresses in detail in the Cognima Server architecture. Here weassume that such a list is available.

A set of one or more Cognima addresses is attached to the object whichis to be shared. The object can be set to read-only (so the people youshare it with cannot modify it). When the Cognima Server receives thenew object (or receives an update to it) from the web or a client deviceit replicates it as normal.

It also looks up the list of ‘sharees’ Cognima addresses. It marks theobject with an originator id (i.e. the Cognima address of the objectowner+the object id) and sends it to the sharees. The sharee users mayexist on the same Cognima Server or be on different Cognima Servers. TheCognima Server of the sharee receives the object. If it is a new objectit assigns a new object id (keeping note of the originator id). If it isan update it finds the object using the originator id. If the sharee isallowed to update the object, the update can be replicated back to theobject owner using the originator id.

9. Displaying Data

Conventional small devices like PDA tend to have simple filing systemsthat allow applications to read and write data to some kind of storagethat will keep the data when the application is not running. Generallythese programs will tend to read in the available set of data and thenprovide a user interface to display the data on the screen. This hassome disadvantages:

Reading in the data when the program starts takes time

The application needs to store all or some of the data in memory meaningit is now occupying more memory on the client device

Allowing more than one application to access the same set of databecomes non-trivial

Similar code to read and manipulate the data appears in severalapplications that run on the device.

The Cognima approach is different:

Data is stored in an Object Database that can be accessed by severalapplications

A Cognima application does not read in all the data it deals with from adatabase. Instead it creates a selection—a subset of the data which itis currently interested in. In general this selection matches the datathat is currently being displayed on the devices screen. Thus only thedata currently being used by the application is held in memory—saving alot of memory space.

All of the work of storing, sorting and indexing the data is done by theObject Database and so this functionality does not need to be repeatedin each application.

When changes need to be made to data in an application, the applicationnever directly updates its own display of the data. Changes will updatethe properties in an object or create or delete an object. A change tothe data could be made by another application or an update received froma Cognima Server due to the data being changed on another machine.

When an application sets up a selection it gives a list of criteria bywhich data is either included or excluded from the selection—because ofthis the Cognima Replication Engine can tell which applications tonotify when a object is created, deleted or updated.

When an update needs to be sent to the application, code in theapplication linked to the selection that contains this data is calledand in this way the application can respond to the changes that havebeen made.

When selections are set up, the application can also specify how thedata is sorted and if only a small window on the sorted list of data isrequired (known as a view).

This approach is similar to the screen re-paint approach used to redrawgraphics screens on Windowing systems. When an area of the screen needsrepainting the application that is responsible for that bit of screen iscalled to repaint the screen.

9.1 Example

A client device may have a contacts application running on it—thisdevice replicates data with a Cognima Server connected to other clientdevices also running contacts applications. A class of object is definedfor a Contact that contains names and phone numbers and these arereplicated to all the devices of a given user.

An application on one device may have a display that shows all contactsby beginning letter—for example the interface allows the user to press aD button to show all the names beginning with D. This application willset up a selection that contains objects:

Where the class is defined as Contacts

Where the name begins with the selected letter (e.g. D)

When the selection is defined the application also defines code to becalled by the CRE when objects are added, deleted or updated.

When the selection is first set up this code will be called back withthe first set of objects that fulfil the above criteria.

If the application was asked to create a new contact with a namebeginning with D the application would create the object but do nothingelse. The CRE would detect the new object and call back the selectioncode to notify it of the new object.

Likewise is a new Contact object was created on another device and wasreplicated to the client device—if the name of that Contact began with Dthe application would be notified.

9.2 Sorting

Data in selections generally needs to be sorted—often so that whendisplayed users can see data in a logical format. When a selection isdefined the sorting order can be specified: the properties to sort on,in what order and what sorting algorithms to use.

9.3 Views

There may be many items of data in a selection. Commonly when the datais being displayed it may not all fit on the screen and so the user willneed to scroll up and down the data. A view provides this functionalityby specifying the number of items of data the selection wants to dealwith and the number of the first item of data out of the complete listof data the application wants to appear in the selection.

Views are important because they allow an application to limit theamount of data it stores locally to be limited to just the amount neededto display on the screen this reducing unnecessary duplication of data.

9.4 Efficiency

Cognima has made some efficiency optimisations in how the data istransferred between the Cognima server and client application—whenmultiple data changes are made the data is sent in blocks and then theapplication informed that the changes are complete so that theapplication only needs to update its user interface once.

9.5 Example

As an example we will define a selection called ContactSelection. Thisis the code that the framework will call back whenever a change is madeto any of the selected objects. In the Cognima framework this isimplemented as an object which you derive from the COdbSelect templatedclass—specifying the type of object you want to have in the selection asthe template argument. class CContactSelect : publicCOdbSelect<CContact> { public:  CContactSelect(COdb *aOdb);  voidObjectAdded(CContact *aObject);  void ObjectUpdated(CContact *aObject); void ObjectRemoved(const TOdbObjectId aObjectId); private:  boolListContacts( ); };

The methods ObjectAdded( ), ObjectUpdated( ) and ObjectRemoved( ) arecalled by the framework whenever respectively an object is added,updated or removed. When you implement the Selection class you don'tneed to implement all these methods if you do not want to take instanceaction on any of these events—in some cases you may set up a selectionto keep a list of a certain set of objects but only check that list onsome other event and so the above methods would not be required.

We have defined one extra private method called ListContacts( )—thiswill list au the current contacts held by the selection.

Here is the implementation of this class: CContactSelect::CContactSelect(COdb *aOdb)  :COdbSelect<CContact>(aOdb)  {  }   voidCContactSelect::ObjectAdded(CTestContact *aContact)   {  OdbLog(OdbLogApp,L“New contact added: ” << aContact->   GetName( ));  ListContacts( );  }  void CContactSelect::ObjectUpdated(CTestContact*aContact)  {  OdbLog(OdbLogApp,L“Contact updated: ” <<aContact->GetName( ));  ListContacts( );  }  voidCContactSelect::ObjectRemoved(const TOdbObjectId aObjectId)  { OdbLog(OdbLogApp,L“Contact deleted - Id: ” << aObjectId); ListContacts( ); } void CContactSelect::ListContacts( ) { OdbLog(OdbLogApp,L“Contacts list:”);  for (unsigned long Index=0;Index<iResult.size( ); Index++)  {     CTestContact*Contact=iResult[Index];     OdbLog(OdbLogApp,Contact->GetName( ) << “,”      << Contact->GetPhone( ) << “,”       << Contact->GetEmail( ));   } }

The constructor simply calls the default COdbSelect constructor. TheObjectAdded( ), Updated( ) and Removed( ) methods print out what changewas made and then call ListContacts( ) to show what the current contentsof the list is.

The ListContacts( ) shows how the current list of object held by theselection can be accessed. The current list of pointers to objects isheld in a container class called iResult—this can then be accessed bynormal container class integrators. In this case we simply go throughthe list and print all the objects in the list.

1. Method of automatically replicating data objects between a mobiledevice and a server, connected together via a wireless network, in whichthe timing of data replication across the network is determined by anetwork operator applying parameters that make efficient usage ofnetwork bandwidth.
 2. The method of claim 1 in which the parametersapplied to a given object are both time dependent and also relate to howurgently that object needs to be replicated.
 3. The method of claim 1 or2 in which a change log lists all objects at the device and/or server tobe replicated and the parameters then comprise a weight associated witheach object that defines how urgently that object needs to bereplicated.
 4. The method of claim 3 in which the parameters furthercomprise a threshold that is a function of time, with the weight of eachobject being locally compared to the threshold at a given time and theoutcome of the comparison determining whether the object is sent forreplication or not at that time.
 5. The method of claim 4 in which aconnection is established at a given time if the weight of any objectexceeds the threshold at that time.
 6. The method of claim 3 in whichthe weight of an object at a given time is a function of one or more ofthe following: (a) Direction of data replication (device to server orserver to device) (b) Shelf life, defining the time or duration afterwhich the object will be automatically deleted if still present in thechange log (c) Whether the object is overwritable (d) Size in bytes (e)Time entered into the change log (f) Priority (g) Time out interval (h)Assigned time for replication (i) User assignment of a non-defaultpriority to a given object (j) Memory available
 7. The method of claim 6in which the network operator can cause the weight of an object to bealtered at any time.
 8. The method of claim 4 in which the networkoperator can cause the threshold to be altered at any time.
 9. Themethod of claim 4 in which the threshold varies over time in such a waythat efficient use is made of available bandwidth.
 10. The method ofclaim 4 in which the threshold can vary over time in a different way fordifferent groups of end-users, individual end-users or applications. 11.The method of claim 4 in which dynamic varying of the threshold canoccur as cell or network loadings change.
 12. The method of claim 4 inwhich dynamic varying of the threshold can occur to encourage uptake ofa new data replication service.
 13. The method of claim 4 in which thethreshold can vary depending on one or more of the following: (a)current time (b) device roaming status (c) cell or network loading (d)time since last replication (e) user tariff
 14. The method of claim 4 inwhich, if the weight of no object exceeds the threshold at a given time,the time interval that will elapse before the weight of any objectexceeds the threshold is calculated and a timer set for that timeinterval.
 15. The method of claim 14 in which the time interval isre-calculated if: (a) a new object is added to the change log (b) a newthreshold is deployed (c) the timer expires (d) cell or network loadingalters (e) device memory falls below a predefined level (f) the devicedetects that its roaming state changes (g) a new application isactivated on the device (h) a network connection is terminated
 16. Themethod of claim 1 in which the end-user of the device can overridedefault replication timing in respect of a specific object or type ofobject.
 17. The method of claim 1 in which an object to be replicated isassigned a time limit by which time replication must occur.
 18. Themethod of claim 17 in which the time limit is dynamic.
 19. The method ofclaim 17 in which the time limit alters if memory on the device changesor if the device roams to a new network
 20. The method of claim 1 inwhich an object to be replicated is assigned a shelf life which definesa time or duration after which the object will be deleted automaticallyif not replicated.
 21. The method of claim 1 in which differentparameters enable the network operator to offer end-users differentlevels of data replication service, each associated with a differenttariff.
 22. The method of claim 1 in which, once a connection initiatingobject has been replicated, then further objects in a change log andpending replication are sent as well.
 23. The method of claim 22 inwhich an opportunism threshold function determines the further objectsthat are sent.
 24. The method of claim 23 in which the opportunismthreshold changes if the device is on a roaming network.
 25. The methodof claim 23 in which the opportunism threshold changes depending onwhether a cell loading threshold of the cell the device is located in isexceeded.
 26. The method of claim 23 in which the opportunism thresholdis applied consistently by device and server, with changes to thethreshold communicated across the network.
 27. The method of claim 23 inwhich the network operator can vary the opportunism threshold.
 28. Themethod of claim 1 in which the actual time of replication is a functionof the state of the mobile device, the state of the network and theparameters.
 29. A mobile device programmed with software that enablesthe device to replicate data to a server using the method of claim 1.30. A server programmed with software that enables the server toreplicate data to a mobile device using the method of claim 1.